Application No. 10/629,403 

Reply to Office Action dated: November 12, 2008 

Reply dated: March 12, 2009 

In the Claims: 

Please cancel Claims 33 and 36, and amend Claims 1, 10, 18, 24, and 37-46. Applicant 
respectfully reserves the right to prosecute any originally presented claims in a continuing or future 
application. 

1. (Currently Amended) A method for exchanging information in a shared interactive 
environment, comprising: 

selecting a first physical device in a first live video image wherein the first physical device 
has information associated with it; 

causing the information to be transferred to a second physical device in a second live video 
image wherein the transfer is brought about by manipulating a visual representation of the 
information in the first live video image and in the second live video image : 

wherein at least one of the first physical device and the second physical device has a 
statically or dynamically defined hotspot in the first live video image or the second live video image; 

wherein the manipulation includes interacting with the first live video image and the second 
live video image; 

wherein the first physical device and the second physical device are part of the shared 
interactive environment; and 

wherein the first physical device and the second physical device are not the same. 

2. (Original) The method of claim 1 wherein: 

the manipulation is accomplished by dragging the representation from the first physical 
device and dropping the representation on the second physical device. 

3. (Original) The method of claim 1 wherein: 

changes to the physical devices are visible to all participants in the shared interactive 
environment. 

4. (Original) The method of claim 1 wherein: 

a physical device can include a display, a projector, a printer, a facsimile machine, a 
personal digital assistant, a computer, and a portable computer. 
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5. (Original) The method of claim 1, further comprising: 

annotating at least one of the first live video image and the second live video image. 

6. (Original) The method of claim 5, further comprising: 

automatically transferring the annotation to a physical device if the annotation is at least 
partially drawn over the physical device as it appears in a live video image. 

7. (Original) The method of claim 1 wherein: 

the first live video image and the second live video image are the same. 

8. (Original) The method of claim 1 wherein: 

the information can include a digital file, an annotation, a sound, and an audio/video 
presentation. 

9. (Original) The method of claim 1 wherein: 

at least one of the first physical device and the second physical device has associated with it 
a pop-up control panel through which a user can configure and control it. 

10. (Currently Amended) A method for exchanging information in a shared interactive 
environment, comprising: 

selecting a first object wherein the first object is one of[[:]] 1) a physical device in a first live 
video imagej[;]] and 2) an icon on a computing device; 

causing information associated with the first object to be transferred to a second object 
wherein the second object is the other of[[:]] 1 ) [[a]] the physical device in [[a]] the first live video 
imagej[;]] and [[3]]2) [[an]] the icon on [[a]] the computing device; 

wherein the first physical device has a dynamically defined hotpot in the first live video 

image; 



wherein the transfer is brought about by manipulating a visual representation of the 
information in the first live video image ; 

wherein the manipulation includes interacting with the first object in the first live video image 
and the second object; and 

wherein the physical device is part of the shared interactive environment. 
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1 1 . (Original) The method of claim 1 0 wherein: 

the manipulation is accomplished by dragging the representation from the first object and 
dropping the representation on the second object. 

12. (Original) The method of claim 10 wherein: 

changes to the physical device are visible to all participants in the shared interactive 
environment. 

13. (Original) The method of claim 10 wherein: 

the physical device can include a display, a projector, a printer, a facsimile machine, a 
personal digital assistant, a computer, and a portable computer. 

14. (Original) The method of claim 1, further comprising: 
annotating the first live video image. 

15. (Original) The method of claim 14, further comprising: 

automatically transferring the annotation to the physical device if the annotation is at least 
partially drawn over the physical device as it appears in a live video image. 

16. (Original) The method of claim 1 0 wherein: 

the information can include a digital file, a sound, and an audio/video presentation. 

17. (Original) The method of claim 1 0 wherein: 

the physical device has associated with it a pop-up control panel through which a user can 
configure and control it. 

18. (Currently Amended) A method for annotating a live video image, comprising: 
annotating the live video image; 

automatically transferring the annotation to a physical device in the live video image if the 
annotation is at least partially drawn over the physical device as it appears in [[a]] the live video 
image; and 
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wherein the annotation is visible to at least one participant in a shared interactive 
environment. 

19. (Original) The method of claim 1 8 wherein: 

the at least one participant can interact with the physical device. 

20. (Original) The method of claim 18 wherein: 

each of the at least one participants can interact with the shared interactive environment 
through different computing devices. 

21 . (Original) The method of claim 18 wherein: 

the live video image is one of: a panoramic view and a zoomed view. 

22. (Original) The method of claim 18 wherein: 

the physical device can include a display, a projector, a printer, a facsimile machine, a 
personal digital assistant, a computer, and a portable computer. 

23. (Original) The method of claim 18 wherein: 

the physical device has associated with it a pop-up control panel through which a user can 
configure and control it. 

24. (Currently Amended) A shared interactive environment, comprising: 

a camera system to provide a first live view and a_second live view different from the first live 
view , wherein the second live view can be configured to zoom in on a portion of the first live view; 

a first graphical user interface (GUI) coupled to the camera system and to present the first 
live view and the second live view, wherein the views can capture a physical device; 

a device controller to dynamically control the physical device in response to interaction of a 
first user with the GUI wherein the interaction can including annotating at least one of: 1) the first 
live view; and 2) the second live view ; 

wherein annotations are automatically transferred to the physical device in the live views if 
the annotation is at least partially drawn over the physical device as it appears in the live video 
image; 
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a device tracker coupled to the camera system and to dynamically recognize new physical 
devices; and 

wherein the camera system can be mounted on a mobile, robotic platform. 

25. (Original) The shared interactive environment of claim 24, wherein: 
the first GUI allows the first user to interact the physical device; and 

wherein the interaction of the first user is apparent to a second user via a second GUI. 

26. (Original) The shared interactive environment of claim 24, wherein: 

the device controller can control the physical device through at least one of: 1) an infrared 
communication channel; and 2) one or more networks. 

27. (Original) The shared interactive environment of claim 24, wherein: 

the device tracker can recognize new physical devices by at least one of: 1) image pattern 
recognition; 2) radio frequency transmission; and 3) acoustic signal. 

28. (Original) The shared interactive environment of claim 24, wherein: 
the physical device can be a display; and 

wherein the display can include an image stack. 

29. (Original) The shared interactive environment of claim 25, wherein: 

the first GUI can provide a second live view that is different from the second live view 
provided by the second GUI. 

30. (Original) The shared interactive environment of claim 24, wherein: 
the GUI is implemented as one or more web pages. 

31 . (Original) The shared interactive environment of claim 24, wherein: 

the first user can select the second live view by drawing a diagonal in the first live view. 

32. (Original) The shared interactive environment of claim 24, wherein: 

the physical device has a pop-up control panel that can be made apparent to the first user 
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through the first GUI; and 

wherein the pop-up control panel allows the first user to control and configure the physical device. 

33. (Canceled) 

34. (Original) The shared interactive environment of claim 24, wherein: 

the physical device can be represented by a set of attributes and a set of behaviors. 

35. (Original) The shared interactive environment of claim 34, wherein: 
the representation of the physical device is part of a device hierarchy. 

36. (Canceled) 

37. (Currently Amended) A mach i no computer readable memory mod i um having instructions 
stored thereon that when executed by a processor cause a system to: 

select a first physical device in a first live video image wherein the first physical device has a 
dynamically defined hotspot in the first live video image and wherein the first physical device has 
information associated with it; 

cause the information to be transferred to a second physical device in a second live video 
image wherein the transfer is brought about by manipulating a visual representation of the 
information in the first live video image and in the second live video image ; 

wherein the manipulation includes interacting with the first live video image and the second 
live video image; 

wherein the first physical device and the second physical device are part of the shared 
interactive environment; and 

wherein the first physical device and the second physical device are not the same. 

38. (Currently Amended) The mach i n e computer readable memory m e d i um of claim 37 wherein: 
the manipulation is accomplished by dragging the representation from the first physical 

device and dropping the representation on the second physical device. 

39. (Currently Amended) The mach i no computer readable memory mod i um of claim 37 wherein: 
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changes to the physical devices are visible to all participants in the shared interactive 
environment. 

40. (Currently Amended) The mach i n e computer readable memory m e d i um of claim 37 wherein: 
a physical device can include a display, a projector, a printer, a facsimile machine, a 

personal digital assistant, a computer, and a portable computer. 

41 . (Currently Amended) The mach i n e computer readable memory m e d i um of claim 37 further 
comprising instructions that when executed cause the system to: 

annotate at least one of the first live video image and the second live video image. 

42. (Currently Amended) The mach i n e computer readable memory m e d i um of claim 37 wherein: 
automatically transferring the annotation to a physical device if the annotation is at least 

partially drawn over the physical device as it appears in a live video image. 

43. (Currently Amended) The mach i no computer readable memory mod i um of claim 37 wherein: 
the first live video image and the second live video image are the same. 

44. (Currently Amended) The mach i n e computer readable memory m e d i um of claim 37 wherein: 
the information can include a digital file, an annotation, a sound, and an audio/video 

presentation. 

45. (Currently Amended) The mach i n e computer readable memory m e d i um of claim 37 wherein: 
at least one of the first physical device and the second physical device has associated with it 

a pop-up control panel through which a user can configure and control it. 

46. (Currently Amended) A system, comprising: 

means for selecting a first physical device in a first live video image wherein the first physical 
device has information associated with it; 

means for causing the information to be transferred to a second physical device in a second 
live video image wherein the transfer is brought about by manipulating a visual representation of the 
information in the first live video image and in the second live video image ; 
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wherein at least one of the first physical device and the second physical device has a 
statically or dynamically defined hotspot in the first live video image or the second live video image; 

wherein the manipulation includes interacting with the first live video image and the second 
live video image; 

wherein the first physical device and the second physical device are part of the shared 
interactive environment; and 

wherein the first physical device and the second physical device are not the same. 
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